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Abstract 

Depth of an object concerns a tradeoff between computation time and excess of pro- 
gram length over the shortest program length required to obtain the object. It gives an 

■ unconditional lower bound on the computation time from a given program in absence 
\& | of auxiliary information. Variants known as logical depth and computational depth are 

expressed in Kolmogorov complexity theory. 

We derive quantitative relation between logical depth and computational depth and 
unify the different depth notions by relating them to A. Kolmogorov and L. Levin's fruitful 
notion of randomness deficiency. Subsequently, we revisit the computational depth of 
infinite strings, introducing the notion of super deep sequences and relate it with other 



approaches. 



1 Introduction 

The information contained in an individual finite object (a finite binary string) can be mea- 
sured by its Kolmogorov complexity — the length of the shortest binary program that computes 
the object. Such a shortest program contains no redundancy: every bit is information; but 
is it meaningful information? If we flip a fair coin to obtain a finite binary string, then with 
overwhelming probability that string constitutes its own shortest description. However, with 
overwhelming probability also, all the bits in the string are apparently meaningless informa- 
tion, just random noise. 

The opposite of randomness is regularity; and the effective regularities in an object can 
be used to compress it and cause it to have lower Kolmogorov complexity. Regular objects 
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contain laws that govern their existence and have meaning. This meaning may be instantly 
clear, but it is also possible that this meaning becomes intelligible only as the result of a long 
computation. For example, let the object in question be a book on number theory. The book 
will list a number of difficult theorems. However, it has very low Kolmogorov complexity 
since all theorems are derivable from the initial few definitions. Our estimate of the difficulty 
of the book is based on the fact that it takes a long time to reproduce the book from part of 
the information in it. We can transmit all the information in the book by just transmitting 
the theorems. The receiver will have to spend a long time reconstructing the proofs and the 
full book. On the other hand, we can send all of the book. Now the receiver has all the 
useful information without literally, and does not have to spend time to extract information. 
Hence, there is a tradeoff: in both cases we send the same information in terms of Kolmogorov 
complexity, but in the former case it takes a long time to reconstruct it from a short message, 
and in the latter case it takes a short time to reconstruct it from a long message. The existence 
of such book is itself evidence of some long evolution preceding it. The computational effort 
to transform the information into 'usable' information is called 'depth'. 

We also use a central notion in Kolmogorov complexity: that of 'randomness deficiency'. 
The randomness deficiency of an object in a particular distribution quantifies the 'typicality' 
or 'randomness' of that object for that distribution. A randomness deficiency of tells us that 
the object is typical (we believe that the object is randomly drawn from the distribution). A 
high randomness deficiency tells us that the object is atypical and not likely to be randomly 
drawn. Finally, we consider the information in one object about another one and vice versa, 
and since these are approximately equal we call it 'mutual information.' 

Results: For finite strings, we derive quantitative relations between the different notions 
of depth: logical depth and computational depth (Section 3). In Section 4 we prove that these 
two notions of depth are instances of a more general measure, namely, Levin's randomness 
deficiency, i.e., computational depth is the randomness deficiency with respect to the time 
bounded universal semimeasure and logical depth is the least time for wich the randomness 
deficiency with respect to the time bounded apriori probability is upper bounded by the 
significance level. 

Next, we study the information contained on infinite sequences. Applying the randomness 
deficiency with respect to M M, where M is the universal lower semicomputable semimea- 
sure over {0,1}°°, Levin [Lev 741 [Lev84j defined mutual information for infinite sequences. 
We observe that despite the correctness of the definition, it does not fully achieve the de- 
sired characterization of mutual information. For example, if a = a 1 a 2 ... and 7 = 7 1 7 2 ... 
are two Kolmogorov random sequences and we construct the sequence (3 = a^a 2 ^ 2 ..., then 
I{a : pi) = I(P : a) = 00. However intuitively P has more information about a than the other 
way around since from P we can fully reconstruct a but from a we can only recover half of p. 
In order to fulfil our intuition we propose some definitions of normalized mutual information 
for infinite sequences. We relate this notion with the constructive Hausdorff dimension, using 
the result proved by Mayordomo in May02|. Namely, we show that the normalized mutual 
information of a with respect to P is at least the ratio of the constructive Hausdorff dimen- 
sions of a and P up to an additive factor that measures the difficulty to recover the initial 
segments of a from the initial segments of the same size of p. This connection motivates the 
definition of dimensional mutual information for infinite sequences. This measure, contrarly 
to the normalized mutual information, is symmetric and it is at most the minimum between 
the normalized mutual information of a with respect to P and vice versa. 

In the last section we revisit the notion of depth for infinite sequences, proposing a new 
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depth measure called dimensional depth. As the name suggests, this measure is related to the 
constructive Hausdorff dimension. We prove that dimensional depth is at most the difference 
between time bounded and resource unbounded versions of constructive Hausdorff dimension 
and finally we fully characterize super deepness using our proposed measures in a similar way 
as done in |JLL94j . 

Previous work: Bennett (Ben88j introduced the notion of logical depth of an object as 
the amount of time required for an algorithm to derive the object from a shorter description. 

Antunes et al. AFMV06] consider logical depth as one instantiation of a more general 
theme, computational depth, and propose several other variants based on the difference be- 
tween a resource bound Kolmogorov complexity measure and the unbounded Kolmogorov 
complexity. 

For infinite sequences, Bennett identified the classes of weakly and strongly deep sequences, 
and showed that the halting problem is strongly deep. Intuitively a sequence is strongly deep 
if no computable time bound is enough to compress infinitely many of its prefixes to within 
a constant number of bits of its smallest representation. An interpretation of strongly deep 
objects is given in [LL99] ; a strongly deep sequence is analogous to a great work of literature 
for which no number of readings suffices to exhaust its value. Subsequently Judes, Lathrop, 
and Lutz [JLL94] extended Bennett's work defining the classes of weakly useful sequences. 
The computational usefulness of a sequence can be measured as the class of computational 
problems that can be solved efficiently, given access to that sequence. More formally, for 
infinite sequences, a sequence is weakly useful if every element of a non-negligible set of 
decidable sequences is reducible to it in recursively bounded time. Lathrop, and Lutz [JLL94] 
proved that every weakly useful sequence is strongly deep in the sense of Bennett. Later, 
Fenner et al. [FLMR05] proved that there exist sequences that are weakly useful but not 
strongly useful. Lathrop and Lutz }LL99] introduced refinements (named recursive weak 
depth and recursive strong depth) of Bennett's notion of weak and strong depth, and studied 
its fundamental properties, showing that recursively weakly (resp. strongly) deep sequences 
form a proper subclass of the class of weakly (resp. strongly) deep sequences, and also that 
every weakly useful sequences is recursive strongly deep. 

Levin [Lev 74} ILev 84] showed that the randomness deficiency of x with respect to /i is the 
largest, within an additive constant, randomness /i-test for x. So 5{x | /i) is, in a sense, a 
universal characterization of "non-randomness", "useful" or "meaningful" information in a 
string x with respect to a probability distribution \x. 



2 Preliminaries 

We briefly introduce some notions from Kolmogorov complexity, mainly the standardize no- 
tation. We refer to the textbook by Li and Vitanyi |LV97j for more details. Let U be a fixed 
universal Turing machine. For technical reasons we choose one with a separate read-only 
input tape, that is scanned from left-to-right without backing up, a separate work tape on 
which the computation takes place, and a separate output tape. Upon halting, the initial 
segment p of the input that has been scanned is called the input "program" and the contents 
of the output tape is called the "output". By construction, the set of halting programs is 
prefix free. We call U the reference universal prefix machine. In the rest of this paper we 
denote the n- length prefix of an infinite sequence a by a n and the ith bit by a 1 . 
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Definition 2.1 (i) The (prefix) Kolmogorov complexity of a finite binary string x is denned 
as 

K(x) = min{|p| : U(p) = x}, 
v 

where p is a program, and the Universal a priori probability of x is 

Qu(x)= £ 2 HP| - 

U(p)=x 

(ii) A time-constructible function t from natural numbers to natural numbers is a function 
with the property that t{n) can be constructed from n by a Turing machine in time of order 
0(t(n)). For every time-constructible t, the t-time-bounded Kolmogorov complexity of x is 
defined as 

K l {x) = min{|p| : U(p) = x in at most t{\x\) steps}, 
p 

and the t-time bounded Universal a priori probability is defined as 

qUx)= Yl 2 ~ bl > 

U t (p)=x 

and = x means that U computes x in at most t(\x\) steps and halts. 

A different universal Turing machine may affect the program size \p\ by at most a constant 
additive term, and the running time t by at most a logarithmic multiplicative factor. The 
same will hold for all other measures we will introduce. 

Levin [Lev 74] showed that the Kolmogorov complexity of a string x coincides up to an 
additive constant term with the logarithm of 1/Qu(x). This result is called the "Coding 
Theorem" since it shows that the shortest upper semicomputable code is a Shannon- Fano code 
of the greatest lower semicomputable probability mass function. In order to state formally 
the Coding theorem we need the following theorem on the existence of a universal lower 
semicomputable discrete semimeasure (Theorem 4.3.1 in |LV97| ). 

Theorem 2.2 There exists a universal lower semicomputable discrete semimeasure over {0, 
denoted by m. 

Theorem 2.3 (Coding Theorem) For every x € {0, l} n , 

K(x) = — log Qu(x) = — logm(x) 

with equality up to an additive constant c. 

Hence, if x has high probability because it has many long descriptions then it must have 
a short description too. 

We refer to mutual information of two finite strings as 

I(x:y) = K(x) + K(y)-K(x,y). 

Notice that the mutual information is symmetric, i.e., I(x : y) = I(y : x). 
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3 Depth 



Bennett [Bcn88| defines the 6-significant logical depth of an object x as the time required by 
the reference universal Turing machine to generate x by a program that is no more than b 
bits longer than the shortest descriptions of x. Bennett talks about time as the number of 
steps; without loss of generality we consider the number of steps where t is a time- 

constructible function. 

Definition 3.1 (Logical Depth) The logical depth of a string x at a significance level b is 



where the minimum is taken over all time constructible t. 

Given a significance level b, the logical depth of a string x is the minimal running time 
such that programs running in at most t(\x\) steps account for approximately a l/2 b 
fraction of x's universal probability. This is Bennett's Tentative Definition 0.3 in [Bcn88] p. 



In fact, with some probability we can derive the string by simply flipping a coin. But for 
long strings this probability is exceedingly small. If the string has a short description then 
we can flip that description with higher probability. Bennett's proposal tries to express the 
tradeoff between the probability of flipping a short program and the shortest computation 
time from program to object. 

Antunes et al. [AFMV06] developed the notion of computational depth in order to capture 
the tradeoff between the amount of help bits required and the reduced computation time to 
compute a string. The concept is simple: they consider the difference of two versions of 
Kolmogorov complexity measures. 

Definition 3.2 (Basic Computational Depth) Let t be a time constructible function. 
For any finite binary string x we define 



In Definition 1 of [Ben88] p. 241 we find 

Definition 3.3 A string x is (i(|x|), 6)-deep iff t{\x\) is the least number of steps to compute 
x from a program of length at most K(x) + b. 

Then, it is straightforward that depth*(x) = K l {x) — K (x) iff x is (i ( ) , K l {x) — K(x))-deep. 
Bennett remarks, |Bcn88j p. 241, "The difference between [Definitions 13.31 and 13. 1| is rather 
subtle philosophically and not very great quantitatively." This is followed by [Bcn88j Lemma 
5 on p. 241 which is an informal version of [LV97] Theorem 7.7.1. The proof of Item (ii) 
below uses an idea in the proof of the latter theorem. 

Definition 3.4 Let t be a recursive function. Define K (t) as the (prefix) Kolmogorov com- 
plexity of t by K{t) = minjji : Tj computes t(-)}, where Ti, T2, ... is the standard enumeration 
of all Turing machines. 




240. 



depth* (x) = K\x)-K(x). 
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Theorem 3.5 Let t be a time-constructible function (hence it is recursive and K{t) is defined 
in Definition \3.1$ . 

(i) If b is the minimum value such that ldepth b (x) = i(|x|), then depth*(x) > b + 0(1). 

(ii) // depth* (x) = b, then ldepth b+min{K{b)K{t)}+0{1) (x) > t(\x\). 

Proof, (i) Assume, ldepth b (x) = t(|x|). So 

Qu( x ) > 
Qu{x) ~ 

with t{\x\) least. Assume furthermore that b is the least integer so that the inequality holds 
for this i(|x|). We also have 

Qll( x ) > 2 ~ K ^ _ 2-{K t (x)-K(x)~0(l)) _ 2 -6-A 

Qu(x) - Qu(x) 

where b+ A = K t (x) — K(x) — 0(1). The first inequality holds since the sum Q\j{x) comprises 
a term 2~ K ^ based on a shortest program of length if* (a;) computing x in at most t(\x\) 
steps. Since b is the least integer, it follows that A > 0. Since depth* (x) = K l {x) — K(x), we 
find that depth*(x) > b + O(l). 

(ii) Assume that depth*(x) = b, that is, x is (t(|x|), 6)-deep. We can enumerate the set 
S of all programs computing x in time at most t(\x\) by simulating all programs of length 
I < |x| + 21og|x| for i(|x|) steps. Hence, the shortest such program q enumerating S has 
length |g| < K(x,t) + 0(1). But we achieve the same effect if, given x and b we enumerate 
all programs of length I as above in order of increasing running time and stop when the 
accumulated algorithmic probability exceeds 2~ K ( x )+ b . The running time of the last program 
is t(\x\). (This shows that K(t,x) < K(b,x) + O(l), not K(t) < K(b) + O(l)). The shortest 
program r doing this has length \r\ < K(x,b) + 0(1). Hence, K(S) < mm{K(x, t), K(x, b)} + 
O(l). By definition, Q\j{x) = S P g5^~' p '. Assume, by way of contradiction, that 

Qll( x ) < 2-b-mm{K(b),K{t)}-0(l) 

Qu(x) 

Since Qjj{x) = 2- R ^-° ( - 1 \ we have 

Q\j{x) < 2 -^( ;E )- b - min {^(b)^W}-0(i) 

Denote m = K(x) + b + min{K(b), K(t)} + 0(1). Therefore, Y. P as 2 ~ H < 2 ~ m - Now ever y 
string in S can be effectively compressed by at least m — K(S) — 0(1) bits. Namely, 

^ 2 -|p|+m K 1 

pes 

The latter inequality is a Kraft inequality, and hence the elements of S can be coded by a 
prefix code with the code word length for p at most \p\ — m. In order to make this coding 
effective, we use a program of length K(S) to enumerate exactly the strings of S. This takes 
an additional K(S) + O(l) bits in the code for each p E S. This way, each p £ S is effectively 
compressed by m — K(S) — 0(1) bits. Therefore, each p € S can be compressed by at least 
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K(x) + b + min{K(b), K(t)} — mm{K(x, t), K(x, b)} bits, up to an additive constant we can 
set freely, and hence by more than b bits which is a contradiction. Hence, 



Qu( x ) > 2-b-mm{K(t),K(b)}-0(l) 

Qu(x) ~ 



which proves (ii). 



4 A Unifying Approach 

Logical depth and computational depth are all instances of a more general measure, namely 
the randomness deficiency of a string x with respect to a probability distribution, Levin 
|Lev741 IL"ev84] , In the rest of this paper, with some abuse of notation (sec [LV97]), a function 
H : {0, 1}* — ► R defines a probability measure, or measure for short, if 

Me) = 1, 

ae{0,l} 

Definition 4.1 Let fi be a computable measure. The value 

Qu(x) 



5(x | /j,) 



log- 



H(x) 



is the randomness deficienc^ of x with respect to fx. Here Qjj is the universal a priori 
probability of Definition 12.11 



Note that Qu{x) is of exact order of magnitude of 2~ K ( X > by the Coding Theorem 12.31 
i.e., up to multiplicative terms Qu(x) and 2~ K ( X ' are equal. (In the literature, see for ex- 
ample |LV97| . m(x) = 2~ K ^ is used instead of Qu(x), and it is straightforward that this is 
equivalent up to a multiplicative independent constant by the Coding Theorem.) 

We now observe that logical depth and computational depth of a string x equals the 
randomness deficiency of x with respect to the measures Q t (x) = Ylu t (p)=x 2 ~' p ' an d ^ K ^ 
respectively. The proofs follow directly from the definitions. 

Lemma 4.2 Let x be a finite binary string and let t be a time-constructible function. 

(i) ldepth b (x) = minjt : S(x \Q l ) <b}. 

(ii) depth* (x) = 5{x \ m*) where m t {z) = 2~ K W . 



5 On the information of infinite strings 

Based on the unification of depth concepts for finite strings, in this section we extend those 
ideas for infinite sequences. In order to motivate our approach we start by introducing 
Levin's notion of randomness deficiency for infinite sequences. Let M be the universal lower 
semicomputable (continous) semimeasure over {0, 1}°° as defined, and proved to exist, by 

1 |_t-J denotes the integer part of r and \a] denotes the smallest integer bigger than a. 



7 



|Lev84j (see also [LV97j ). If a G {0,1}°°, then with a = a x a 2 ... with a { G {0,1}, we 
write a n = a a 2 . . . a n . Finally, we write 'M(x)' and '^(x)' as notational shorthand for 
'M^)' and ^(T x )\ with x G {0, 1}* and T x is the cylinder {uj : uj G {x}{0, 1}°°}. Strictly 
speaking, M(x) is not over {0,1}°° but over {0, 1}°° \J{0, 1}*, see also |LV97j . and M(x) is 
the probability concentrated on the set of finite and infinite sequences starting with x. 

, ( M(a n ) 
log sup 



Definition 5.1 (Levin) The value D(a I //) = log I sup — -, — r- I is called the random- 

L \ « M a n) / J 

ness deficiency of a. with respect to the semimeasure [i. Here M(a n ) is the probability 
density function of M(a„). 

Let a and (3 be two sequences and M ® M be defined by M <g> M(a, 0) = M(a)M(/3). 

Definition 5.2 (Levin) The value 7(a : 0) = D((a,@) \ M <8> M) is called the amount of 
information in a about (3 or the deficiency of their independence. 

This definition is equivalent to the mutual information I {a : (3) = sup n /(a n : j3 n ). 



Example 5.3 Let a and 7 be two random infinite and independent strings (in the sense that 
their prefixes are independent). Consider the following sequence 

P = aVaV • • ■ 

By Definition 15.21 we have 

I(a : p) = 

> 

As I(P : a) = I (a : P) then J(/3 : a) = 00. 

However, intuitively P contains more information about a than the other way around, 
since from the sequence p we can totally reconstruct a but from a we can only recover half 
of P, namely, the bits with odd indexes. 

This seems to be a lacuna in Definition [5T2J The definition says more when the information 
is finite but that is precisely when we do not need an accurate result. Notice that if the 
sequences are finite we can argue that they are independent. In the infinite case, one should 
be able to classify the cases where the mutual information is infinite. Two infinite sequences 
may have infinite mutual information and yet infinite information may be still lacking to 
reconstruct one of them out of the other one. In the previous example a fails to provide 
all the information of P related to 7, which has infinite information. In this section we will 
present two approaches to reformulate the definition of "mutual information" in order to fulfill 
our intuition. In order to have a proportion of information as the prefixes grow we need to 
do some normalization in the process. 



sup/(a n : p n ) 

sup(K(a n ) + K(p n ) - K(a n ,p n )) 

n 

sup [n + n — (ji + ^ J ^ =00. 
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5.1 The Mutual Information Point of View 

We are looking for a normalized mutual information measure I m that applied to Example 
gives 

I m (a : a) = 1 
I m (a : p) = 1/2; 
I m (P : a) = 1 
I m (P ■ P) = 1 

Contrarily to Levin's definition of mutual information for infinite sequences, and accord- 
ingly to our intuition, the above conditions imply that the normalized version must be non- 
symmetric. 

Definition 5.4 (First attempt) Given two infinite sequences a and (3 the normalized mu- 
tual information that P has about a is defined as 

I m (P : a) = lim lim ' ""^ 

moorrnoo ±{tt n \ OL n ) 

The major drawback of this definition is the fact that the limit does not always exist H 
However, it does exist for the Example 15.31 with the desired properties. Furthermore, we 
obtain for the same a and (3 

I m (a : a) = 1; 
I m (P :P) = 1; 

T , n s ,. ,. m + n - (m + n - n/2) 1 
I m (a : P) = hm hm = -; 

n->oo m->oo n Z 

I m (P ■ a) = lim lim — — = 1; 

n— +oo m^oo n 

Definition 5.5 (Normalized mutual information for infinite sequences) Given two 
infinite sequences a and P we define the lower normalized mutual information that P has 
about a as 

I m *(P '■ a) = liminf lim }f m — ^4 

n^oo m-*oo l[ot n \ OL n ) 

and the upper normalized mutual information that P has about a as 

t* t a \ r r I\Pm '■ a n) 
l m {p : a) = lim sup lim — - 

n^oo m~>co l(a n \ Ct n ) 

Notice that these definitions also fulfill the requirements presented in the beginning of this 
section with respect to Example 15.31 

We now can define independence with respect to normalized mutual information: 

Definition 5.6 Two sequences, a and P, are independent if Im{ a '■ P) = ^nSfi ■ a ) = 0- 



2 Notice that there are sequences a for which lim — ; — - does not exist. 

n K{a n ) 
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In [LutOCH ILut02| . the author developed a constructive version of Hausdorff dimension. 
That dimension assigns to every binary sequence a a real number dim(a) in the interval 
[0, 1]. Lutz claims that the dimension of a sequence is a measure of its information density. 
The idea is to differentiate sequences by non-randomness degrees, namely by their dimension. 
Our approach is precisely to introduce a measure of density of information that one sequence 
has about the other, in the total amount of the other's information. So we differentiate 
non-independent sequences, by their normalized mutual information. 

Mayordomo |May02| redefined constructive Hausdorff dimension in terms of Kolmogorov 
complexity. 

Theorem 5.7 (Mayordomo) For every sequence a, 

dim(a) = lim inf — an 

So, now the connection between constructive dimension and normalized information mea- 
sure introduced here is clear. It is only natural to accomplish results about the Hausdorff 
constructive dimension of a sequence, knowing the dimension of another, and their normalized 
information. 

Lemma 5.8 Let a and (5 be two infinite sequences. Then 
I^(a : 13) ■ dim(/3) > dim(a) + lim inf 



Proof. 



Iinipc : (5) ■ dim(/3) = lim sup lim — m — • lim inf — [E]±L 

n m I(fj n : fj n ) n n 

> lim inf lim inf - m — 

n m n 

> lim inf lim inf ^• OLm — 



n m rtl 

. . ,,. . , K(a m ) - K(a m \(5„ 

> lim inf lim mf ■ 



m rn 
. . ,K{a rn ) . -K(a m \p n 

> iim mf h lim mi ■ 



mm rn m 

= dim(a) + lim inf ( a m\Pm) 

m m 

o 

Note that, in the previous lemma the (unexpected) aditive term lim inf ( a m\Pm) . g 

m m 

necessary to expresses the hardness of recover a given f3. 

We present now the time bounded version of dim(a). This definition will be important 
later on this paper. 

Definition 5.9 Let t be a time-constructible function. The i-bounded dimension of an infi- 
nite sequence a is defined as 

dim* (a) = lim inf 

n^oo n 
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5.2 The Hausdorff constructive dimension point of view 

In this subsection we define a version of mutual information between two sequences based on 
Hausdorff constructive dimension and establish a connection to it. 

Definition 5.10 The dimensional mutual information of the sequences a and (5 is defined as 

Idim( a '■ 0) = dim(a) + dim(/3) — 2 dim (a, ft) 

This measure of mutual information is symmetric. The definition considers twice dim (a, (3) 
because when encoding the prefixes a n and (3 n the result is a 2n-length string. Notice that, 



'■dim 



a:j3) = dim(a) + dim(/3) — 2 dim (a, f3) 

= lim inf — ^ + lim inf — —jz— — 2 lim inf ■ 



n/2 n— >oo n/2 n-*oo n 

K(a n/2 ) + K((3 n/2 ) - K(a n/2 ,P n / 2 ) 



< lim inf 

n— >oo n/2 



lim inf l^lM 

n— >oo 77, 

I(a n : (3 n ) 



< lim inf 

< urn mf lim 



n— >oo m— >oo 



= I m *(a : /3) 
The third inequality is true due to the following fact: 

I(Pn : am) = K((3 n ) - K((3 n \a m ) > K{(3 n ) - K{(3 n \a n ) = I{(5 n : a n ). 

By the symmetry of the definition we also have that Idimi® '■ P) < Im*{/3 '■ a). These two 
facts prove the following lemma: 

Lemma 5.11 Let a and (3 be two sequences. Then 

Idim{a ■ (3) < min(/ m *(Q : /3),i" m *(/3 : a)) 

One can easily modify the definitions introduced in this section by considering the limits 
when n goes to the length of the string, or the maximum length of the strings being considered. 
One should also notice that when x and y are finite strings and K(y) > K(x), I m *(x : y) is 
1 — d(x,y), where d(x,y) is the normalized information distance studied in [Li03j. 



6 Depth of infinite strings 

In this section we revisit depth for infinite sequences. We introduce a new depth measure, 
prove that it is closely related with constructive Hausdorff dimension and use it to characterize 
super deepness. To motivate our definitions we recall the definitions of the classes of weakly 
(vs. strongly deep) sequences and weakly useful (vs. strongly useful) sequences. 

Definition 6.1 (|Ben88j) An infinite binary sequence a is defined as 
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• weakly deep if it is not computable in recursively bounded time from any algorithmically 
random infinite sequence. 



• strongly deep if at every significance level b, and for every recursive function t, all but 
finitely many initial segments a n have logical depth exceeding t(n). 

Definition 6.2 ([FLMR05]) An infinite binary sequence a is defined as 

• weakly useful if there is a computable time bound within which all the sequences in a 
non- measure subset of the set of decidable sequences are Turing reducible to a. 

• strongly useful if there is a computable time bound within which every decidable se- 
quence is Turing reducible to a. 

The relation between logical depth and usefulness was studied by Juedes, Lathrop and 
Lutz [JLL94] who defined the conditions for weak and strong usefulness and showed that 
every weakly useful sequence is strongly deep. This result generalizes Bennett's remark that 
the diagonal halting problem is strobgly deep, strengthening the relation between depth and 
usefulness. Latter Fenner et al. [FLMR05] proved the existence of sequences that are weakly 
useful but not strongly useful. 

The Hausdorff constructive dimension has a close connection with the information theories 
for infinite strings studied before, see for example [FLMR05], [LutOO] . [Lut02] and |May02| . 
Therefore, in this section we define the dimensional computational depth of a sequence in 
order to study the nonrandom information on a infinite sequence. 

Definition 6.3 The dimensional depth of a sequence a is defined as 

5(a n I 2~ Kt( - an ^) 
depth dim (a) = liminf . 

Lemma 6.4 

depth^^a) < dim* (a) — dim(a) 

Proof. 



depth*^ (a) = liminf 



5(a n | 2~ K 



n— >oo n 
n^oo 77, 

< dim* (a) — dim(a). 

The last inequality holds since the sequence of values K(a n )/n is non negative and then 

liminf —K(a n )/n < — liminf K (a n )/n. o 
n n 

Now, in the definition of strongly deep sequences, instead of considering a fixed significance 
level s we consider a significance level function s : N — » N . Naturally, we want s(n) to grow 
very slowly so we assume for example that s = o(n). With this replacement we obtain a 
tighter definition as deepness decreases with the increase of the significance level. 

Definition 6.5 A sequence is called super deep if for every significance level function 
s : N — > N, such that s = o(n), and for every recursive function t : N — > N, all but finitely 
many initial segments ot n have logical depth exceeding t(n). 
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We have already characterized super deep sequences using their dimensional depth in 
Theorem 13.51 In fact we have 

ldepth 6 (a;) = with b minimal =>■ depth*(x) > b + O(l) 

Theorem 6.6 A sequence a is super deep if and only if depth*j im (a) > for all recursive 
time bound t. 

Proof. Let a be a super deep sequence. Then for every significance level function s, such that 
s = o(n) and every recursive function t we have that for almost all n, ldepth s ( n )(a n ) > t(n). 
Then 

depth t(n) (a«) > s(n). 

Now if for some time bound g, depthj im (a) = then there exists a bound S, such that 
S = o(n), and, infinitely often 

depth^Vn) < S(n). 

This is absurd and therefore for all recursive time bound t, depthj im (a) > 0. 

Conversely if depthj im (a) > then there is some e > such that for almost all n, 
depth^(a n ) > en. This implies that 

ldepth s( - n -)(Q n ) > ldepth en (a ra ) > t(n) 

for all significance function s = o(n) and almost all n. So a is super deep. o 

In the next theorem we express other equivalent ways to define super deepness. 

Theorem 6.7 For every sequence a the following conditions are equivalent. 

1. a is super deep; 

2. For every recursive time bound t : N — > N and every significance function g = o(n), 
depth* (a n ) > g(n) for all except finitely many n; 

3. For every recursive time bound t : N — > N and every significance function g = o(n), 
Q(o-n) ^ 2 5 '( n )Q i (a n ) for all except finitely many n; 

Proof. [Sketch] The equivalence (1 2) was proved in Theorem 16.61 To show that (2 3) 
consider the following sets: 

D\ = {a E {0, 1}°° : depth*(a n ) > g(n) a.e.} 

D\ = {a G {0, 1}°° : Q(a n ) > 2^Q\a n ) a.e.} 
The proof nows is an immediate consequence of the following lemma: 

Lemma 6.8 (Lemma 3.5 in [JLL94]) Ift is a recursive time bound then there exists con- 
stants c\ and Co and a recursive time bound ti such that D*\ „, C Dt and C Dt. 
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o 

Following the ideas in |JLL94| to prove that every weakly useful sequence is strongly deep 
we can prove that every weakly useful sequence is super deep. 

Theorem 6.9 Every weakly useful sequence is super deep. 

For the proof of this result we need the following lemmas: 

Lemma 6.10 (Lemma 5.5 in [JLL94]) Let s : N — > N be strictly increasing and time- 
constructible with the constant c s as witness. For each s-time-bounded Turing machine M, 
there is a constant cm that satisfies the following. Given non- decreasing functions t,g : N — > N 
we define s*, r, t, g : N — > N by 

s*{n) = 2*(P°s n l)+ 1 , 
r(n) = t(s*(n + 1) + 4s*(n + 1) + 2(n + l)c,a(|w|) + 2ns*(n + l)s(M)), 
i= c M (l + r(n) [logr(n)l), 
g = g(s*(n + 1)) +c M , 

where w is the binary representation of n. For all sequences a, (5, if (3 is Turing reducible to 
a in time s by M and (3 £ Dt then a £ D g . 

Lemma 6.11 (Corollary 5.9 in [JLL94]) For every recursive function t : N — > N and 
every < 7 < 1, the set has measure 1 in the set of recursive sequences. 

Proof, [of Theorem 16.9] Let a by a weakly useful sequence. To prove that a is super deep we 
show that for every recursive time bound t and every any significance level g = o(n), a £ D g , 
where D g is the set defined in proof of Theorem 16.71 

Since a is weakly useful then there exists a recursive time bound s (that without lose 
of generality we can assume increasing) such that the set DTIME a {s) of all sequences that 
are Turing reducible to a has positive measure in the set of recursive sequences. Using 
Lemma 16.101 to conclude that a £ D g all that is necessary is to prove that there exists 

f3 G D| D DTIME a (s), where t and g are described in same lemma. 

Fix 7 G]0, 1[ and consider t{n) = n(l + j(n) [log n]) where 7 is obtained from t and s 
as in Lemma 16.101 Since t is recursive, by Lemma 16.11} L>* n has measure 1 in the set of all 
recursive sequences. Thus D^ n f]DTIME a (s) has measure 1 and in particular is non empty. 
As t(n) > t{n) a.e. and 771 > o(n) = g a.e. it follows, directly from the definitions, that 
-D* n C Dt and then L>* n / 0, as we wanted to show. o 

Corollary 6.12 The characteristic sequences of the halting problem and the diagonal halting 
problem are super deep. 

Proof. In [Bcn88j , the author proved that the characteristic sequences of the halting problem 
and the diagonal halting problem are weakly useful. Then, it follows from Theorem 16.91 that 
these two sequences are super deep. o 
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